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Abstract. In this paper, we consider a class of multi-particle reinforced inter- 
acting random walk. In this model, there are some (finite or infinite) particles 
performing random walk on some (finite or infinite) graph, so that each par- 
ticle has higher probability to visit neighboring vertices or edges which have 
been seldom visited by the other particles. Specifically in this paper we in- 
vestigate two particles' vertex-reinforced interacting random walk on finite 
complete graph. By a dynamical approach we prove that the two particles' 
occupation measure asymptotically has small joint support almost surely if the 
reinforcement is strong. 



1. Introduction 

"Reinforced random walk" (RRW) is a remarkable example of non-Markovian 
random walk, which has been extensively studied in the last twenty years, see the 
survey [16]. A good story to help understand RRW appeared in [17]. Imagine a 
person getting acquainted with a new city. She walks about the area near the hotel 
somewhat randomly, but tends to traverse the same blocks over and over as they 
become familiar. To model this, RRW is defined on the vertices of an undirected 
graph in such a way that the probability of a transition from one vertex to an- 
other depends on the number of previous transitions along the neighboring edges 
(respectively, vertices). We refer to such models as "edge- reinforced random walk" 
(ERRW) (respectively, "vertex-reinforced random walk" (VRRW)). 

RRWs can be defined on all kinds of graphs, either finite or infinite. It also can be 
defined by very different reinforcement mechanisms through the weight functions, 
which could lead to quite different phenomena to occur. A question of interest is 
the recurrence or transience of RRW. For example, Angel et al. [1] showed that 
the linearly ERRW on any graph with bounded degrees is recurrent for sufficiently 
small initial weights. Another phenomenon of interest is localization. For exam- 
ple, Pemantle and Volkov [18] showed that VRRW on Z has finite range, and then 
Tarres [21] showed that VRRW on Z eventually gets stuck on five points. Phase 
transitions are also of interest. For instance, Pemantle [14] showed that the ERRW 
on infinite binary tree can vary from transient to recurrent, depending on the value 
of an adjustable parameter measuring the strength of the feedback (reinforcement), 
and Volkov [22] showed that VRRW on Z with weight function k a will just visit 
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2, 5, oo sites after a large time To when a>l,= lor<l respectively. 

So far, almost all the considered RRW models are one particle' self-interacting 
with positive (attractive) feedback (reinforcement), i.e. the edges (or vertices) 
already being visited more are more likely to be visited in the future. In analyz- 
ing such models, four main methods are commonly used: exchangeability [9] |16) . 
branching process embedding [6] jlOJ [11] |20j . stochastic approximation via mar- 
tingale methods [7] [T5], and dynamical system approach [5] [I]. Here we want to 
say several more words about the history of the dynamical system approach, which 
has been used in the first half of this paper. The dynamical system approach is an 
approach to analyze stochastic approximation process (defined later) by its approx- 
imating ODE. This approach has been extensively studied since it was introduced 
by Ljung [12]. From the 90s, Benai'm and his collaborators have developed this 
approach, which enables to analyze stochastic approximation process with much 
more general dynamics. 

One direction to generalize RRW is to consider multi-particle RRW. Kovchegov [9] 
showed for the two particles' edge reinforced process on Z, the two particles meet 
infinitely often a.s.. In his model, each particle's transition probability is positively 
reinforced (determined) by both particles' visits on the edges. Except his paper, 
the generalization of RRW to multi-particle RRW models with more general rein- 
forced interacting mechanism hasn't appeared yet so far. In 2010, Itai Bcnjamini 
proposed a class of new multi-particle reinforced interacting random walk, and fi- 
nally our paper is a result of exploration on this model. 

The organization of this paper is as follows: Section 2 is devoted to the definition 
of the model; Section 3 is devoted to state our main result; Section 4 is the proof 
of main result; Some further open problems are proposed in Section 5. 

2. The model 

In this section, we define a class of two-particle vertex-reinforced interacting 
random walk on a connected graph. Denote the two particles by X and Y, and 
the graph by G — (V,E). At each time unit, both X and Y particle will jump 
to some vertices on V. Set X, Y's locations at time k on V as random variables 
Xk, Yfc, and the number of X, Y's visits to vertex v by time n as random variables 
N(X,v,n), N{Y,v,n). In our model, we assume X, Y visit each vertex initially. 
Then we have 

n n 

N(X,v,n) = l + Y / ^x k =v, N(Y,v,n)=N(Y,v,0)+Y,lY k =v, Vu € V. 

k=l fc=l 

Denote the natural filtration generated by {Xk, < k < n} and {Y&, < k < n} by 
J- n {n S N). Then the stochastic process (X n , Y n )'s transition probability is defined 

F(X n+1 = v\T n ) = h=*<> • ^-"> (2.1) 

Lp~X n w N(Y,p,n) 

and 

nY n+1 = v\r n ) = ]r Y >>' w w\ (2.2) 

^P~Y n W N {X,p,n) 
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where "~" generally represents some vertex relation on G (e.g nearest-neighbor), 
and Wk(k £ Z+) is a fixed sequence of positive numbers, referred to as "weights". 
Like the classical one-particle RRW, one natural weight sequence to work with is 
Wk = k~ a for some a > 0. 

On a finite graph G of size No, one can set 

*,(„) = ffii^, „ W ^,», £ y (2.3, 

n + N n + No 

as X and Y's empirical occupation measure on V by time n. Then the 2Nq di- 
mensional vector [(i„(n))„ E y, (y v (n)) V £v] is a random process living in a bounded 
subset of Euclidean space E 2JV ° . If Wk (like k~ a ) is homogenous in k, we can rewrite 
(|2~Tj) and (gUJ) as 

P(X n+ i = uf,, = = (2.4 

and 

nY n+1 =v\f n )= ^-™ff» (2.5) 

where w(x) is a function vj : [0,1] — > R + . Hence, X,Y's transition probabilities 
are defined by their occupation measure. 

In this paper, we will work with the above defined model (|2.4p and (I2.5[) on finite 
complete graph. The vertex relation "~" is that the graph distance between two 
vertices is less than or equal to 1 and zu(x) = [81 x <$ + a;la:>i] ° for some small 
positive 5. For convenience, we label the vertex set V of a finite complete graph of 
size d by the set S — {1, . . . , d}. Then for any i, j <= S, (|2.4[) and (|2.5[) becomes 

P(X„ +1 = W = [^W^+^W^n)^- ^ (2 . 6) 

Lfc=i [ l ' 1 to(i.)<J + !/M") 1 MW>jJ 

and 

[^lxj(«)<5 + a; j( rl )l2; 3 (n)>5] 
Yfk=l [ S1 x k (n)<6 + Xk(n)l Xh ^n)>s] 

Set x(n) = (xi(n))igs, y(n) = (j/ J (n)) lG s and z(n) = (x(n),y(n)). Then in this 
paper we will prove z(n)'s asymptotic behavior. Notice that z(n) is a Markov 
chain living in M. 2d . 

Here we want to mention that when minj j 6 <j{a!j(n), yj(n)} > 5, (|2.6p and (I2.7[) 
are equivalent to the following formulas: 

F(X n+1 =j\F n )= N(Y ' J 'f~\ VyeS; (2.8) 

F(Y n+1 =j\T n )= N{ X' 3 > n ^ a , ViJeS, (2.9) 
Ek=iN(X,k,n)— 

which can be easily seen as X, y's repelling interacting random walk with nonlinear 
reinforcement. 

Notation 2.1. (1) In this paper, we always write A as the (ef— 1)— dimensional 
closed simplex and denote the interior of A by A°; 
(2) Denote the product of two simplex (A x A) by D, which is the compact 
domain in our model. Denote the interior of D by D° ; 



F{Y n+1 - j\J- n ) - — — — — r-r^, [Z./) 
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(3) We will use x (or y,u,v) to represent vectors in A and index the time 
parameter n or t in the vectors by writing x(n) or x(t). The coordinate 
i(€ S) of the vectors are indexed like Xi(n) or Xi(t). We write (x,y) ( or 
(it, v) ) to represent a 2d dimensional vector, i.e. {x,y) = z = (2i)i<i<2d 
where 2j = £,(1 < i < d) and = yi-d{d + 1 < * < 2d); 

(4) When a 2<i dimensional vector (a;, y) is the variable of some function, say 
F, we will write F(x,y) instead of F((x,y)) for simplicity; 

(5) In this paper, we let || • || denote the L 1 norm on M. 2d or M. d . 

3. The main result 

For the same <5 appearing in w(x), we define S s = {(x, y) G D : Y^i=i x iVi < 
2rf 1 /(«+ 1 )(5}. Notice that 2c? 1 ^ Q+1 ^ is a constant independent of S. Then we have 
the following theorem for any fixed S > 0. 

Theorem 3.1. When a is large enough, the random process z(n) 's two components 
x(n) and y(n) asymptotically have small joint support bounded by cS almost surely, 
i.e. 

F13N, fl {z(n)eS s } 1=1. 

[ n>N J 

Remark 3.2. In this paper, for any fixed d > 3 G N. we say a is large enough, if 
the following two conditions hold: 

(1) Proposition \A.1\ holds for this a; 

(2) 4-3g^ > 0, i.e. a > logd/log|. 

The second condition is needed in the proof of Lemma \^.12\ 

4. Proof of the main result 

In Subsection 14.11 we will conclude that z(n) belongs to a class of stochastic 
approximation algorithms. In Subsection 14.21 we will introduce the interpolated 
process which has the same limit set as z(n). In Subsection 14.31 we will introduce 
the deterministic ODE associated with z(n) and construct a non-rigorous Lyapunov 
function. In Subsection 14.41 we will state Benaim's theorem which addresses the 
relation between the interpolated process and the ODE. In Subsection 14.51 we will 
give our main lemma, which describes the so-called chain-recurrent set for our 
specific semiflow induced by the ODE. In Subsection 14.191 we will use Pemantle's 
theorem to show that z(n) has probability to converge to some isolated unstable 
equilibrium. In Subsection 14.71 we finally give the proof of Theorem 13.11 

4.1. Stochastic approximation algorithms. 

Definition 4.1. Stochastic approximation algorithms are discrete time stochastic 
processes whose general form can be written as 

£(n + 1) - £(n) - v n H{Z(n), e(n)) (4.1) 

where H : measurable function that characterizes the algo- 

rithm, {£(n)}„>o G K m is the sequence of parameters to be recursively updated, 
{e(«-)}n>o G R d is a sequence of random inputs where H(£(n),e(n)) is observable, 
and {a(n)} n >o is a sequence of "small" nonnegative scalar gains. 
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Stochastic approximation algorithm basically is a discrete time random process 
defined by a recursive formula. The theory of stochastic approximations was born 
in the early 50s with the works of Robbins and Monro and Kiefer and Wol- 
fowitz [8]. 

Now to show that that z(n) is stochastic approximation algorithm, we start with 
(fHo]) and (f2J]) . Notice that the right hand sides of ([2~6]) and ([2~7) is a map M : 
D — > D and this map is continuous on the whole compact D, but not differentiable 
near the boundary of D (denoted by dD) according to the definition of -cu(x). Set 
a map n = {■niji^s : A — > A: 

n(x) = -^—, VxgA. (4.2) 

2^=1 M^fc) 



Then we can derive a difference equation for Xi(n) 
x l {n + 1) - Xi(n) 



N(X,i,n) + l Xn+1 =i N(X,i,n) 



n + l + N n + N 

Xi(n) + l Xn+1 =i 



n +1+N 



n+l + N n + l + N 

Similarly, we can derive a difference equation for yi(n). Set 



(4.3) 



^- 'ntr+T" eX(,i) = <eX(,,)) -« <4 ' 4) 



and 

It follows from JUJ and dSTTJ) that 

E(ef (n))|J-„) = 0, E(ef(n))|J-„) = 0, Vz € 5. (4.6) 

Then we can write the difference equation for the random process (x(n),y(n)) in 
the following vector form: 

x(n + 1) - z(n) = - - ^ + — {-x{n) + 7r(j/(n))) + e X (n) (4.7) 

j/(n + 1) - y(n) = 1 (-y(n) + 7r(z(n))) + e Y (n). (4.8) 
n + 1 + N 

Sete(rc) = (e x (n), e Y (n)) and F(z(n)) = F(x(n),y(n)) = (-x(n)+Tr(y(n)), -y(n)+ 
n(x(n))). One can also write (|4.7I) and (|4.8|l as 

z(n + 1) - z(n) = n+1+jVo ^W) + e(n). (4.9) 

Then (14. 9|) implies that z(n) is stochastic approximation algorithm. The stochastic 
approximations in the form of (|4.9j) with ¥,(e(n)\J- n ) = is also called Robbins- 
Monro stochastic approximation algorithm. So our random process z(n) is Robbins- 
Monro stochastic approximation algorithm. 
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As mentioned before, we will use the dynamical system approach to analyze 
(|4.9|) . The dynamical system approach mainly says one can introduce the averaged 
ordinary differential equation (denoted by ODE) 

d ^=F{z{t)) (4.10) 

and then describe the asymptotic behavior of (14.91) by the semiflow (solution) in- 
duced by flUTDJ) . 

Remark 4.2. In general, the derivation of \4 . 1 0[ ) from the stochastic approxima- 
tions is more complicated. See [2J. 

An application of the dynamical system approach in VRRW is given in [3J. By 
the limit set theorem proved in [2J, the main difficulty to apply this approach 
usually boils down to analyze the deterministic ODE, which is nontrivial in many 
occasions. Lyapunov function method is an ideal way. However, there is no general 
method to construct such a function. 



4.2. The interpolated process. In this section, we define stochastic approxima- 
tion algorithm z(n)'s interpolated process. Set tq = 0,o~ n = X/(k + No),T n = 

En 
k=l a k- 

Definition 4.3. z(n)'s interpolated process is defined as a continuous-time random 
process z(t) : R+ — > D which satisfies: 

(1) *(*)lt=r„ = *(»); 

(2) z{t) is affine on [t„,t„ + i]. 



The interpolated process in Definition 14.31 can be written in equations as follows: 



*(*) - E 



( Z (n + l)- z{ n )) l/{ ^ No) +Z (n) 



l[r„,r„ +1 )W (4-H) 



Set z(t) = (x(t),y(t)) where x(t),y(i) € A. For any fixed t € [r n ,r„+i), from (|4.7I 
and (|4.1ip . we can write the interpolated process of x(n) as follows 



x(t) = (x(n + 1) - x(n)) 
= (x(n + 1) — x(n)) 



(t - T n ) 



l/(n + 1 + N ) 
(* - r n ) 



x{n) 

n 

J2(x(k) - x(k - 1)) + x{0) 



k=l 



l/(n + 1 + N ) 
[{-x{n) + n(y(n))) + (n + 1 + N Q )e x (n)] (t - r„) 

71 

+ [(~x{k - 1) + vr(y(fc - 1))) + (k + N )e x (k - 1)] (r fe _! - r fc ) + x(0). 

(4.12) 



fc=i 



Notation 4.4. For any sequence {w(n)} n > 6 M. d , denote by w°(-) the step func- 
tion 

(4.13) 
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Set 7(71) = (ri + 1 + N )e x (n)(n > 0). Then according to Notation [OJ (gUl) 
can be written in integral forms 



a?(t)-a;(0)= / (-x°(s) + ir(y°{s)))ds + / 7°(s)ds. (4.14) 
Jo Jo 

Let rj(n) = (n + 1 + ATo)e y (n) (n > 0). Similarly we have 

y(t) - 2/(0) = / H/°00 + + f T)°( S )ds. (4.15) 



Set*°(i) = (x°(t),y°(t)) and9°(t) = (j (t) , rj° (t)) . Then we can rewrite the integral 
equations (|4. 14[) and (|4.15l) in a more unified form 

z(t)-z(0)= f F(z°(s))ds+ f 9°(s)ds. (4.16) 
Jo Jo 

Let 

L ({z(n)} n > ) = p| {z(n) :n>k}, (4.17) 

fe>0 

and 

L({z(t)} t > ) = f){z(t):t>s} (4.18) 

s>0 

denote the omega limit set of z[ri) and z(t) respectively. Then we have the following 
lemma due to Benai'm. 

Lemma 4.5. [H Lemma 4.1] L {{z(n}} n >o} = L ({z(*)}t>o)- 

4.3. The deterministic ODE. Set f(x) = Sl x <s + xl x> g, i.e. f(x) = zu~i. 
According to the above analysis, the deterministic (approximating) ODE in our 
case is: 



<*"i(*) _ fr\ | /(»») ° A 



(4.19) 



or in vector form 



[-^- = -v(t) + 7T{u(t)) dt 

where H(t) = (u(t),v(t)) G D. 

We call the right hand side of (|4~1"9"|) and (|4T2D|) as the ODE's vector field. In 

our case, the vector field F is defined as a map F : A x A — > T(A x A), i.e. 
F : D -> TD, where 



TD = 



T(Ax A)= J( U ,t;)eK 2d ,^ Ul = 0,^ = ol 
I ies ies J 



Notice that the function /(x) satisfies f(x) > 5 and 1/ f(x) > 1. Hence by the fact 
that composition of Lipschitz functions is Lipschitz, F is globally Lipschitz. By 
theorem of maximal interval of existence of ODE's solution (see e.g. [13l Theorem 
3.17]), it follows that the vector field F generates a unique semiflow in D 

$ : JR + x D ->• D, 
(t,o) -> $t(a), 
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such that the solution to the initial value problem dE(t)/dt = F(S(t)) with initial 
condition H(0) = a 6 D is the curve t — > <&t(a). Here to make things clear, we give 
the following definition. 

Definition 4.6. A semiflow $ on a metric space (D,/i) is a continuous map 

$:l + xfl4fl 

(t,x) -> $(t,x) = $ t (x) 

such that 

$o = Identity, $ f+s = $ t o $ s 
for all (i, s) € R+ x R + . Replacing R + by R defines a flow. 

Definition 4.7. A set J is said to be invariant (respectively, positive invariant) 
under a flow * if for all i € K (respectively, for all t > 0), *t(7) C I. 

In this paper, we always work with semiflow. 

Proposition 4.8. TTie domain D is positively invariant under i/ie dynamics of 
F in gjjfy . 

Proof. Suppose (u, v) £ dD. Without loss of generality, we can assume that there 
exists some i € S such that ui = 0. Then by the definition of (|4.19p . we have 



dui{t) 



dt 



>mf ;w- a >0, 



— a 



Hence, F(u, v) points inward whenever (u, v) belongs to the boundary of D. Thus 
any forward trajectory based in D remains in D. □ 

Notation 4.9. In this paper, we always use U to represent the d dimensional 
vector (1/d, ...,1/d). According to the previous notation, it is clear that (U,U) 
represents a 2c? dimensional vector with each coordinate equal to 1/d. Sometimes, 
we call U as uniform distribution. 

It is easy to check that p — (U, U) is a critical point of the vector field F, i.e. 
F(p) = 0. After the following definition, we will classify what type of critical 
point it is. Let T be the linear approximation to F near a critical point p so that 
F(p + w) = T(w) + 0(M 2 ), then 

Definition 4.10. If all the eigenvalues of T have strictly negative real part, p is 
called an attracting point; if some eigenvalues of T have strictly positive real part, 
p is called an unstable critical point. 

Proposition 4.11. (1) When a > 1, (U,U) is a linearly unstable critical point 
for F. 

(2) When a < 1, (U, U) is an attracting point for F . 

Proof. Recall dE(t)/dt = F(E(t)) in (|4.20l) . By Taylor expansion of F in a neigh- 
borhood of (U, U), we have 

F(p + w) = DF\-w + 0(\w\ 2 ), 
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where DF is Jacobian matrix and z is some vector in a neighborhood of (2d 
dimensional vector). By direct calculation, one can have 



DF\ 



' -1 












a: 









-1 







a 
ci 




a 

d 












-1 


a 
rf 




a 
d 




-«+f 


a 
d 




a 
d 


-1 










a 
d 


-« + ! 


a 
,/ 







-1 







a 
d 




a 
d 












-1 



\ 



V 

In order to get all the eigenvalues of DF| , one need to solve 

DF\ p - \I 2d x2d = <>• 



(4.21) 



where | • | represents the matrix's determinant and I2dx2d is a 2d dimensional identity 
matrix. Notice that DF\ 1 
We set this d x d matrix as 



matrix. Notice that DF\ has the same right-upper and left-lower block matrix. 



B 



\ 



-«+§/ 



Because the sum of B's each row is zero, \B\ = 0. Then one can easily check A = — 1 
is a solution to (|4.21[) . Now assume A ^ -1. By Schur complement, from (|4.21j) we 
get 

|B 2 -(A + l) 2 / rfxd | -0, (4.22) 
where Idxd is a d dimensional identity matrix. (|4.22l) is equivalent to 



\B-(X + l)I t 



dxd\ 



\B + (X + l)I dxd \ =0. 



Under the assumption A ^ —1, we easily get all the solutions to (I4.23[) : A 
Hence all the eigenvalues of DF\ without counting multiplicities are 



(4.23) 
-l±a. 



■l,-l±a. 



So if a > 1, DF\ has the positive eigenvalue — 1 + a; If a < 1, all of its eigenvalues 
are strictly negative. This concludes the proof. □ 

Define the function H (u, v) = Xh=i u i v ii (( w i v ) G D). In fact, H(u, v) plays the 
role of a Lyapunov function in our proof. However, from Lemma 14.121 below, one 
can see that it is not monotone in the whole domain, and hence it is not exactly 
a Lyapunov function according to the classical definition. Set the function H(t) 
associated with the semiflow $ induced by (|4.19[) as 



H(t) = H(u(t), v(t)) = Ui{t)vi(t). 



(4.24) 
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Then 

dH 
~dt 



dui(t) 



5>(*> dt 
i=i 

d / 



z=l 



Cfoj ft) 



•E' 



-2E 



f/,r. 



E?=i«i/0 



EUnf(vi)- a 



(4.25) 



ELi /(«*)-" ELi/W- Q 

Notice that (|4.19l) and (|4.24|) are autonomous, so is (|4.25[) . Hence, the right side 
of (|4.25|) is often considered as a continuous function of (it, v) on £>, which is 
independent of time t. We have the following lemma about (|4.25l) . 

Lemma 4.12. When a is large enough, for any (u,v) G D satisfying H(u,v) > 
5, the derivative of H(t) satisfies 

dH 



dt 



< 0. 



(4.26) 



(u,v) 



In the set £ D : H(u, v) > §e£ a+1 s\, dH /dt = holds if and only if (u, v) — 

(U,U). 

Proof. This proof is divided into three cases: 

(1) min ie s f(ui) > S and min ieS /(«<) > S; 

(2) minigg f{ui) = 5 and muijgs f( v i) > 25 (or the symmetric case); 

(3) min^gg f(v,i) = 5 and mim e s f(vi) < 25 (or the symmetric case). 

In Case (1), min lS s > * and min ieS /(«») > <5 imply /(«j) = and 

/(wj) = Vi for any i £ S. By Proposition ! A. 1 1 in the Appendix, we have showed that 
in this case dH/dt\^ u , < 0, and the equality holds if and only if (u,v) = (U, U). 

In Case (2), from the property of f(x), i.e. f(x) > x and (|A.4|) in the Appendix, 
it follows that 



dH_ 
~dt 



UiV-i 



< 



< 



i=l 
d 

-2 mi). 



Eti"i/("i)~ a , Eti^/fa)" a 
Eti/K)- Eti/K)- 

Eti/(^)/W- Q , E*= 1 /(«0/(«i)- a 



i=l 
d 



ELi/K)" Q 



E fc =i /M~ 



-2Vii,o, (min/(u, i ) +min/(u i ) 



Since 



i=l 



(4.27) 



> u,i>i > max < minui, mini), > 

2 — 1 V ^ 



in Case (2) it follows that 

d 



EUjVi > max < minu ? -,minw.i > > mini), = min f(vi). 
\ ies " jes 3 J - 3 es J ies J y ' 

1—1 V ' 
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Hence, 

d , 
< -2V UiVi + d~ I min /(it*) + min f(vi) 

* i£ S i£ S 



dH 
~dt 



< -2 min f(vi) + d"+ 1 [ S + mmf(vi) 
= - (2- d^^mmf(vi) + d^T5 

< -(2-d^)28 + d^S = -U-3d^^j5. 

So one can choose a large enough such that 4 — 3d'^+ T > 0, obtaining dH/dt\^ u v \ < 
0. 

In Case (3), from the given condition H (u, v) = Ylt=x u i v i > f ^"^^ and (|4.27l) . 
it follows that 



dH 
~dt 



< ~2^2u t Vi + 3d^TT(5 < 0. (4.28) 

(«,«) i=i 



□ 



4.4. Bena'im's theorems. The exact relationship between the deterministic ODE 
and the asymptotic behavior of the interpolated process is addressed by the limit 
set theorem in [2]. We start a definition introduced in [5]. 

Definition 4.13. z(t) is called an asymptotic pseudotrajectory of $ if for T £ R + 

lim sup d(z(t + h),$ h (z(t))) = 0. 

t-»oo <h<T 

Then we have the following lemma. 

Lemma 4.14. The interpolated process z{t) in 16\ ) is almost surely an asymp- 
totic pseudotrajectory of the semiflow $ induced by F in \^.2(ty , i.e. for any 
T E M.+ , 

lim sup \\(z(t + h)-$ h (z(t)))\\=0. 

t->oo < h < T 

Proof. The main part of this proof is extracted from the proof of [U Lemma 4.2]. 

With the same initial condition z(t) 6 D, if comparing the interpolated process 
z(s)'s integral equation (|4.16p with the semiflow $ induced by the ODE (|4. 19[) or 
(jlllOll in the time interval [t, t + h](0 < h < T), we have 

z(t + h) - ® h (z(t)) = z(t)+ / F(z°(t + s))ds + / 9°(s)ds 



z{t)+ f H F($ s (z(t)))dsj 

rt+h 

F(z°(t + s))ds- I F{<f> s (z(t)))ds+ / 9°(s)ds. 

(4.29) 
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Set cp(h) = \\z(t + h)- $ h (z(i))\\. By the Triangle inequality and (|429|) . 



(p(h) < L I tp(s)ds+ I \\F(z(t + s))-F(z°(t + s))\\ds + 

r t+h 

< L I Lp(s)ds + Ll \\z(t + s) - z°(t + s)\\ ds + 



t+h 



e°(s)ds 



9°(s)ds 



,(4.30) 



where L is the global Lipschitz constant of F on D. Set m(t) = sup{p 6 JV : t p < t}, 
it follows from the definition of the interpolated process and (14. 16)) that 



\z(t + s)-z°(t + s)\\ = 



t+h 

r m {t+h) 

< v m (t+h)\\F\\D + 



(F(z a ( S ))+e ( S ))ds 

t+h 

6°(s)ds 



(4.31) 



where ||-F||d is uniform (supremum) norm of F on D. Now (|4.30l) and (|4.31[) 
together imply 



i-h / rt+h 

< L <p(s)ds + LT [ a m(t+h) \\F\\ D + / 

JO \ Jr m(t+ 



6°(s)ds 



L <p(s)ds + LTa m(t+h) \\F\\ D + LT 



(t+h) 

t+h 

m(t + h) 



e°(s)ds 





j-t+h 




J 9°{s)ds 








rt+h 


+ 


J 6°{s)ds 







(4.32) 



Now define 



e(n, T) = sup 

{k:0<T k -T„<T+l} 



fc-i 



where e(i) = (e x (i), e Y (i)) are the error terms defined in (|4.4[) and (|4.5[) . It is easy 
to see 



9°{s)ds 



< 2e(m(t),T). 



Vp,q£ [T m (t),T m ( t +T)+i], 
Then it follows from (|4~32)) that 

<p(h)<L I V ( S )ds + LTa m{t+h) \\F\\ D + 2e(m(t) 1 T)(LT+l). 
Jo 

By Gronwall's inequality, one gets 

sup <p(h)<e LT (LTa m{t+h) \\F\\ D + 2e(m(t),T)(LT+l)). 

he[0,T] 

Because <J m (t+h) — > as i — > oo, the proof of the lemma reduces to show 

lim e(n,T) = 0. (4.33) 



Notice that E(e(i)|J r i) = and ||e(i)|| < c/(i + N ) 2 for some positive constant 
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c. These imply M n = y\ —1 e(i) is a martingale with bounded quadratic variation. 
Thus it converges to some finite random variable almost surely. Hence, 



lim sup 

™^°° {k:0<T k -T n <T+l} 



k-1 



i—n 



= 0. (4.34) 



This concludes the proof. □ 

Before stating Theorem 14. 171 we need to give the definition of chain-recurrence. 
Suppose D is a metric space with a metric p. In our case, p is the metric induced 
by L 1 norm. 

Definition 4.15. Let p > and T > 0. A point z is said to be (p, T) recurrent if 
there exist an integer k, some points Z{ in D, and numbers i.;, < i < k — 1, such 
that 

U>T; p(z ,z) < p; p($ ti (zi), z i+ i) < p, i = 0, . . . , k - 1; x = z k . 

z is said to be chain-recurrent, if z is (/?, T) recurrent for any p > and T > 0. 

We denote by Ci?(<f>) the set of chain-recurrent points. The set Ci?($) has the 
property to be closed and invariant. 

Definition 4.16. A subset R C D is called internally chain-recurrent if R is a 
nonempty compact invariant (respectively, positive invariant) set of which every 
point is chain- recurrent for the restricted flow (respectively, semiflow) 

The following theorem, which is due to Benaim, demonstrates the relation be- 
tween the asymptotic behavior of asymptotic pseudotrajectories and the dynamics 
of $. The theorem can be viewed as a deterministic statement. 
Theorem 4.17. [21 Theorem 1.2, Corollary 4.3] Let z(t) be an asymptotic pseudo- 
trajectories o/$. Then L ({z(t)}t>o), defined in \4- 18\ ), is a connected set internally 
chain-recurrent for the semiflow 



4.5. The main lemma. Now it comes to our main lemma to prove Theorem 13. II 
This lemma describes the chain recurrent set for our specific dynamic $. In the 
proof of this lemma, Lemma T4 . 1 2 1 will be heavily used. Recall S s — {(x, y) € D : 
E?=i XiVi < 2d 1 /( Q+1 )^}. 

Lemma 4.18. CR($) cS s U (U, U). 

Proof. By Lemma 14.121 S s is a compact positively invariant set with respect to the 
semiflow $, and H($ t ) is strictly decreasing when $ t locates outside S s U (U, U). 

Let e > be small enough, s.t. Co := 2d 1 /(«+i) 5 _ eQ > ( 3 / 2 ) . . 6. Set 

^ = 2d 1 /( Q + 1 '(5 and C 2 = 1- Then define M z = {(x,y) € D : H{x lV ) < £}(« = 
0, 1, 2). Note that Mi = S s , M 2 = D. Then we are going to prove this lemma by 
the following steps: 

(1) Any chain {zo, Zi, • • • , Zfe} satisfying p($ ti (z%), z l+ i) < p for some U > T 
with z £ Mj is included in Mj(j = 0, 1, 2); 

(2) CRi = CR{$)C\Mx and CR 2 = CR($)n(M 2 \Mi) are positively invariants 
sets; 

(3) CR 2 = (U,U). 
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Proof of of Step 1. Let T > 0. By compactness of the sets Mj(j = 0, 1, 2), it 
follows from Lemma 14.121 that there exists e > such that 

V2€%ff(#r(«)) <(j-e- 

Pick p > such that 

W(z,w) EDx D,fi(z,w) < \H(z)-H(w)\ < e. 

Then it follows that any chain {zo, zi, . . . , Zk} satisfying p(§ti Zi+i) < p for 
some t i > T with z e M 3 is included in Mj(j = 0, 1,2). □ 

Proof of of Step 2. By the invariance of CR(Q) and Mi, it is clear that CR\ 
is positively invariant. 

For some z £ CR 2 , suppose there exists some To > 0, such that $t (z) G M\. 
Then by Lemma I3~T21 there exists some T' > 0, such that for T := T + T', 

H($ T (z)) < Co- (4.35) 

By continuity of there exists some p > small enough, such that 

$t(z ) & M , Vz a eB(z,p). 
By invariance of Mo, it follows that for alH > T 

$tOo)eM , Wz eB(z,p). 

Without loss of generality, p can be assumed to be smaller than the distance between 
M and D\M X (closure of D \ Mi). Then the following is true for all t > T 

(j,($ t (zo),zi) < p zi e Mi. 

Step 1 tells us that any (p, T) chain (z%, . . . , Zk) with z\ € Mi is included in Mi. 
Hence if p < /i(z,Mi) is further assumed, for any finite sequence (zq, Zi, . . . , Zk) 
with ti > T satisfying 

p(z a ,z)<p, n($ ti (zi), z l+ i) < p. i = 0, ...,k-l, 

the following always holds 

(j,($t h (zk),z) > P- 

This implies z cann't be (p, T) chain recurrent, which contradicts the assumption. 
Hence, ® t (CR 2 ) C M 2 \M 1 for all t > 0. By invariance of CR($), CR 2 is positively 
invariant. □ 

Proof of of Step 3. It suffices to show CR 2 C (U,U). For any z G CR 2 , by 
invariance of CR 2 and the non-increasing property of the Lyapunov function H(-) 
along any trajectories in M 2 \ M%, it follows that the limit of H(Q t (z)) exists. Set 

H($ 00 (z))= lim H{$ t {z)). 

t— t+oo 

Hence, for any p £ l>j(z), H(p) = H(^ 00 (z)). Together with invariance of u>(z) (as 
a set), this implies H(-) is constant along trajectories in ui(z). Therefore, lo(z) C 
(U,U), i.e. cj(z) = (U,U). 

Now we will show z = (U,U). Suppose z ^ (U,U), then H(Q t (z)) is strictly 
decreasing in a neighborhood of z. Since H($ t (z)) > H(ui(z)) — H(U,U) = 1/d, 
there exists some e > 0, s.t. 

H{z) >-,+e. 
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Define 

E=( [ we D\H(w) < - d + 1 1 , F = \w G D\H(w) > i + || . 

By compactness oi D\E and the strictly decreasing property of if (•), there exists 
some T, such that for any t > T 

$ t (w)eE, VweD\E. (4.36) 

In particular, (j4.36[) holds for some small ball B(z,p) C F and all f > T, i.e. 

$ t (w)eS, Vw£B(z,p). (4.37) 

By the similar argument as that in the proof of Step 2, one can get that (I4.37P 
contradicts the fact that z is a chain recurrent point. □ 



Now we can finish the proof of this lemma. From Step 2 and 3, we have CR\ C Mi 
and CR 2 C (U, U), and hence CR($) c M x U (17, U) = S 5 U (17, f7). □ 

4.6. Non-convergence to unstable equilibrium. As for z(n)'s non-convergenc 
to p = (U,U) , we are exactly in the setting of the following theorem due to 
Pemantle [T3] . 

Theorem 4.19. T51 Theorem 1] Define a stochastic process 

z(n + 1) — z[n) — a n F(z(n)) + e(n) 

with o~ n being T n — measurable and E(e(n)|7 r „) = and such that z{n) always re- 
mains in bounded domain D. Let p be any point of D with F(p) = 0, let Af 
be a neighborhood of p and assume that there are constants 7 € (1/2,1] and 
ci,C2,C3,C4 > for which the following conditions are satisfied whenever z{n) G J\f 
and n is sufficiently large: 

(1) p is a linearly unstable critical point, 

(2) ci/nf < On < c 2 /n\ 

(3) E((e(n) • #) + |7-" n ) > c^/n 1 for every unit vector 9 € TD, where TD is the 
tangent space to D, 

(4) \\e(n)\\ <c 4 /n\ 

where (e(n) ■ 9) + — max{e(n) • 0,0} is the positive part of e(n) ■ 9. Assume F is 
smooth enough to apply the stable manifold theorem: at least C 2 . Then 

P ( lim z(n) = p) = 0. 

\ti— >oo / 

Before applying Theorem 14. 191 to our case, we need the following lemma, which 
is to check the noise terms e(n) in our case satisfies Condition (3) in Theorem 14. 191 

Lemma 4.20. In a small neighborhood of p = (U,U), there exists some constant 
c > 0, s.t. E((e(n) • 9) + \!F n ) > c/n for every unit vector 9 = (9k)i<k<2d € TD. 
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Proof. Suppose X n +i = i € S,Y n +i = j £ S. Then according to (|4~4|) and (|4.5|) . 
we have 

(n + l + iV o )e(n)-0 = (1 - ^(^(n)))^ - ^ *Vn(v(»))0»n 

+(1 - nj(x(n)))dj + d - ^ Tr p (x(n))9 p+d 

= 6i + 9 j+d - 22 n m (y(n))9 m - ^ ir p (x{n))9 p+d (4.38) 
meS pes 

For any unit vector 9 £ TD, the following holds 

max k > — =. (4.39) 

i<fc<2d (d-l)V2d 

Indeed, since is a unit vector (say ||#| 1^2=1), 

max \0k\> 



l<fc<2d y/2d' 

Now if the following inequality 

max 9h > — 1= ( > 



i<fe<2<i % /2d V (d-l)\/2d 
doesn't hold, there must be 

min 9 k < =. 

l<k<2d y/2d 

Since J2k=i ®k = and X)fe=d+i ^fc = 0' there exists some coordinate 1 < k < 2d, 
s.t. 



(d - l)V2d 

which implies (|4.39|) . 

From (14.391) . without loss of generality, we can assume 9\ = maxi<fc<2d$fc > 
(d-i)V2d - By Sfeid+i ^ = °> there always exists some j £ S, s.t. 9 JO+d > 0. 

Because (x(n), y(n) lives in a small neighborhood of (£/, U), n(x(n)) and 7r(y(n)) 
also live in a small neighborhood of (£/, £/), and hence in (|4.38l) . 

^ TT m (y(n))9 m ~ 0, ir p (x(n))9 p+d ~ 0. 
mes pes 

Therefore, 

(n + l+iVo)E((e(n)-0) + |.F n ) > P(X n+1 = 1, F„+i = j„)0i 

= p(x (1+1 = i)p(r„ +1 - jo)^i 

> c, 

where the last step is by the fact that (x(n),y(n) lives in a small neighborhood of 
(U, U) and hence F(X n+ i = 1) and V(Y n+ i = jo) are bounded from below. □ 



Now we can apply Theorem 14. 191 to our case. 
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Lemma 4.21. When a > 1, for any initial condition (x(0), y(0)) <E D, z(n) in our 
model satisfies 

P f lim z(n) = (U, U)) = 0. 

\n— too / 

Proof. In our case, x(n) and y(n) are occupation measures, and hence z(n) always 
remains in the domain D for all time n E N. We also have E(e(n)|J r „) = 0. 
Hence z(n) is exactly in the class of stochastic process defined in Theorem 14. 1 91 for 
a n = l/(n + l + JV ),7= 1. 

Now we need to check all the conditions in the theorem hold in our model. 
According to Proposition 14. 1 1] p is a linearly unstable hxed point, and hence Con- 
dition (I) holds. It is also easy to see that Condition (2) and (4) hold in our case. 
Lemma [4.201 implies Condition (3). F is also smooth (at least C 2 ) in a large fixed 
neighborhood of p. Since all the conditions in Theorem 14. 191 hold in our model, we 
can apply the theorem and conclude the proof of the lemma. □ 

4.7. Proof of Theorem [37Q 

Proof. It suffices to show the interpolated process z{t) satisfies 

C d 



t>T I. i=l 

since the two random processes have exactly the same limit set according to Lemma 
1431 

Now from Lemma |4 . 1 4 1 and Theorem l4.17[ it follows that the interpolated process 
z(t)'s limit set satisfies 

P(L({z(t)} t > )cCi?($)) = I. 



Then Lemma 14.181 implies 

F(L({z(t)} t > ) CS S U (U,U)) =1. 

By Theorem 14.171 L ({z(t)}t>o) is nonempty connected compact set. Since (U,U) 
is an isolated equilibrium, together with Lemma 14.211 it follows that 

F(L{{z(t)} t > ) CS S ) =1, 

which concludes the proof. □ 



Remark 4.22. Note that in the proof of Theorem \3.1[ we just used the fact that 
L ({z(t)} t >o) C CR($), which is a weaker result than that stated in Theorem ^. 11\ 
i.e. L ({z(t)}t>o) is internally chain recurrent. 

5. Further questions 

Conjecture 5.1. a c = 1 is critical, i.e. for any given initial condition (x(0), 2/(0)) £ 
D, 

(1) when a > 1, there exists some constant c = c(a, d) depending on a and d, 
such that the following holds for any small positive 5, 



3N > n fezi(»)w(»)<c5j 

n>N U=l J 



= I. 
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(2) when < a < 1, the following holds 

p( lim z(n) = (U,U)\ = 1. 
Question 5.2. When 6 = 0, derive the similar result as Theorem \3.1[ 

For our model defined in (12.11) and (|2.2[) , little is known so far. On 1? , we even 
are not able to prove that each vertex will eventually be visited by at least one of 
X, Y almost surely. Another question of interest is 

Question 5.3. When the weight sequence is Wk = kr a for a > 0, what is the 
nearest neighbor interacting random walk X, Y 's behavior on the lattices % d ? How 
does this behavior change as a and d vary? 

Question 5.4. Similar as h2.1\) and A2.2\) , one can define two particle edge-reinforced 
interacting random walk. Then what is its behavior? 

Question 5.5. Similarly defined as 12. 1\) and h2.2\) . if there are more than two 
particles performing reinforced interacting random walk on some finite or infinite 
connected graph, what is the model's asymptotic behavior? 
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Appendix A. Proof of one inequality 

Proposition A.l. For any fixed positive integer d(> 2), there exists some a(d), 
such that when a > a(d), the following inequality holds: 

2 E U ^^ — r- + ^a — r- ( AJ ) 

i=l L^i=\ Z^i=l ^TT3" 

for all {u, v) = ((uj)ie5, (fOies) € D° . In particular, the equality holds if and only 
if Ui = Vi = 2 f or an y h i- e - ( U 7 V ) — (U, U) . 

To prove this proposition, we need several lemmas. 

Lemma A. 2. For any fixed a > and positive integer d, the following holds for 
all aj e (0, = l,...,d- 1), 

f(a u . . . ,a„_i) := ^ ^ a < ^ ■ 
1 + 2^i=i a i 

Proof. By General Mean inequality, for any a > 0, we have 

f i + EttV- V < A + Et"i< +1 ^ 



d 



i.e. 



Therefore, 



d-l / d-l n 

i=l \ i=l / 



/(ai, . . . , a<j-i) < t-- r— n — = — < rfa!+1 • 



□ 



Lemma A. 3. WTien a > d — 2, [/ (uniform distribution) is a local minimum for 
the following function 



g(ui, . . . , Ud) := 2 min Ui — d f minw 



2 
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In particular, g ( -j, 



0. 



Proof. Set = Mj5 > 0. By change of variables in function g(u±, . . . , Ud), we have 

Ed j_ 
i=l wf 

G(w 1 ,...,w d ) = -mm Wl - — (mm Wl ) - - j - 



where 5 = X)i=i w i- Notice that G(w\, . . . , Wd) is a homogeneous function on 
(R d ) + , and Uj = Wi/S 1 = Wi/Y^j=\ w j is a natural bijection between the lines 
in (R d ) + and the points in A°, which gives the exactly same value through the 
functions G{w\, . . . , u>d) and g(u\, . . . ,u d ). So we can work with the function 
G{w\ , . . . , Wd) without constraints instead of g(ui, . . . , Ud) with constraints, to 
prove that the diagonal (w\, . . . , w d ) = (w, . . . , w) (w is any positive real number) 
is a local minimum of G(w\, . . . , w d )- 

Without loss of generality, we assume w d = min ie s Wi, then 

1 ' ""' (A.2) 



G(W1, ...,W d ) = ^W d - -^(Wdf 



X^d 1 



By direct calculation, we can check that G(w\, . . . ,Wd) has zero gradient on the 
diagonal, i.e. 

VGL i=t0 = o. 

Then the Hessian matrix of G(wi, . . . , w d ) evaluated at the diagonal Wi = w is 



(- 



Sct 



M 



and 



q+2 
d 



•a + 1 



a+2 



a+2 

.S±2+ a + l _'«±2 



a+2 



a+2 
d 



a+2 _a+2 +Q , + 1 
d d ^ " ^ 1 

1 _ a+2 

1 d 



a+2 



+ a + l 

a+2 
d 



a+2 

d + " ^ 1 d 



a+2 

v i-¥ 



/ a+2 a+2 
' d d 

a+2 



a+2 
d 



1 _ «+2 
1_ «±2 



^ _ a+2 



a+2 



1 _ «+2 
- 1 - j 



1 _ a+2 
X d 



--±2+ a + l + 2-d/ 



q+2 q+2 
d d 



q+2 
d 



iV = 

\i-¥ 

Then if, M and JV satisfy 
if 



1 d 

q+2 q+2 ^ _ a+2 

d d d 



^ = ^2 M > M=(a + 1)/ + 7V, 
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where / is identity matrix. And by direct calculation, we can get all the eigenvalues 
of matrix N 

Af = ... = Al 2 = 0,Al 1 =-(a + l),Af = - (-^ + 1 
and hence get all the eigenvalues of M (shifted by a + 1 from N's eigenvalues) 

a + 2 

It is easy to see that when a > d — 2, one of M's eigenvalues is zero and all the 
others are positive. It is also easy to check that the sum of each row of M is zero, 
which means 

n 

M : = 

w 

That is, the diagonal is an eigenvector associated with M's zero eigenvalue and 
then £Ts zero eigenvalue. This tells us that our original function G(wi, . . . , Wd) is 
convex along all the other directions except the diagonal, which implies that the 
diagonal is its local minimum. Hence the uniform distribution is a local minimum 
of g(u-L, ...,u d ). □ 

Lemma A. 4. For any fixed positive integer d, there exists some a(d), such that 
when a > a(d), the following inequality holds: 

minu; > — j '- — (A.3) 

for any u — (u\, . . . , Ud) € A°. In particular, the equality holds if and only if u is 
uniform distribution, i.e. u = U . 

Proof. We will prove (| A.3|) in two cases: 

(1) u is far from the uniform distribution, i.e. min^gs u; < n/d for some positive 
k < 1 (close to 1); 

(2) near the uniform distribution. 

In Case (1), we try to use the minimum coordinates of u to bound the right hand 
side of (|A.3[) from above. More precisely, for fixed d, a, we try to find some small 
c(a, d) such that the following inequality holds for any u G A°, 

Y^r ~ 

— -, — — < c(a, d) minu,. (A. 4) 

2^1=1 

Notice that without loss of generality, we can assume Ud = min^s Ui, and then 
if we set ai = min^gs Ui/ui — Ud/ui G (0, 1] in (|A.4j) . the problem to get a small 
constant c(a, d) in (|A.4|) is equivalent to get a good upper bound for the following 
quantity 

f(a 1 ,...,a d . 1 )= tin 1 1 U ( A -5) 

1 "+" 2^i=l a i 



nth ai G (0, l](i = 1, . . . ,d- 
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By Lemma [A. 21 we will take c(a, d) = d a + 1 . Then it is easy to check that when 
a > logG?/log(2 — k) — 1, the following inequality holds 

d^Ti . Ud < 2u d - du d , i.e. d~^ < 2 - du d (A. 6) 

for any < u d < n/d. Then the inequalities (|A.4|) and (|A.6[) together imply that 
when a > \ogd/ log(2 — k) — 1, 



ies \ ies 



2 



2miniti — d mint^ I > — -j 5 — (A-.7) 



Ed 1 
i=l ^ 



— 



for any u satisfying min^s ui < n/d. 
In Case (2), Lemma [A. 31 implies (|A.3j) . 

Finally, we need to glue the results in the above two cases together. For fixed pos- 
itive integer d, when a > d—2, there exists a neighborhood of uniform distribution 
N(U,e a ) s.t. 

S(ui,...,u«i)>0 (A.8) 

for any u g Af(U,e a ). The equality holds if and only if u = U. For any u ^ U 
in A°, since Y^t=i iWSiLi is a decreasing function in a (can be proved by 
directly checking the derivative), so g(u±, u d ) is an increasing function in a, 
which allows us to take some common Af(U, e d ) — f~l Q >d— 2 -^"(^ e a) j us t depending 
on d such that the inequality (|A.8j) holds. Now we take some k = n(d) < 1 such 
that 

u e A° : minu, < - t } U j\f(U, e d ) = A°, 
ies d J 

and then combine the inequalities (|A.7[) and (|A.8|) . obtaining that when 

f , logd 
a > max < a — 2, 



log(2 - K(d)) 

(|A.3p holds. The equality condition can also be easily checked. □ 

Proof of Proposition I A. T] First for any u, v € A°, we will bound the left hand 
side of inequality (|A.1[) from below by the minimum coordinates of u and v. More 
precisely, if we set 



and 



we will show 



minu.;, . . . , minui, 1 — (d — 1) min Ui I 

\ies ies ies ) 



1 — (d — 1) minu,;, min«i, . . . , min 

ies ies ies 



^UiVi^^u'iv'i, (A.9) 



i=l i=l 
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where 

d 



y v!{a' i — (d — 2) min Uj min v t + 1 — (d — 1) min Ui I min Vi 



1 — (ci — 1) min Vi min u , 

min Ui + min Vi — d ■ min min 

i65 ieS ies iGS 



is a function just depending on the minimum coordinates of u and v. By the 
Rearrangement inequality, it suffices to show (IA.9I) is true for any increasing {u{\i^s 
and decreasing {v z }i<=s in A°. For such {u.^^s and {vi} ie s, one can have 



^^uiVi = + Ujv d + infa - 

i=l 2=2 i=2 

d d 

i=2 i=2 

d 

= - (d - + ^ UjVd 



I'd 



I'd ) 



i=2 

d d 
i=l i=l 

where the last step is obtained by repeating the same argument as in the previous 
steps. Hence, we have proved (|A.9|) . 

By Lemma lA. 41 we know that there exists some a{d), such that when a > a{d), 
the following inequalities hold for any u, v £ A° 

Y^d 1 

2miniti — d ( minu.; ) > 

and 



( ■ V 



Ed 
1=1 



7+^ 



2 ^ d _L 

2 min Vi — d ( min w< > 



Then by the following inequality 



Ed 
i=l 



2 min Us + min Vi — d min Hi min u, 
> 2 min u% — d\ min Ui I + 2 min v* — d { min u, 

\ iGS \i£S J I \ iES \i£S 

we have proved (|A.1[) . 

It is also easy to check the equality holds if and only if both u and v are uniform 
distributions, i.e. u = v = U. □ 
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